ESTIMATING THE EXPECTED PREDICTIVE ACCURACY OF ECONOMETRIC MODELS BY RAY C. FAIRI A method is proposed in this paper for estimating the uncertainty of a forecast from an econometric model. The method accounts for the four main sources

A method is proposed in this paper for estimating the uncertainty of a forecast from an econometric model. The method accounts for the four main sources of uncertainty: uncertainty due to (1) the error terms, (2) the coefficient estimates, (3) the exogenous-variable forecasts, and (4) the possible misspecification of the model. It also accounts for the fact that the variances of forecast errors are not constant across time. Because the method accounts for all four sources of uncertainty, it can be used to make accuracy comparisons across models. The method has two advantages over the common procedure of computing root mean squared errors (RMSEs) to evaluate the accuracy of econometric models. The first is that the RMSE procedure does not account for the fact that the variances of the forecast errors vary across time. Although RMSEs are in some loose sense estimates of the averages of the variances across time, no rigorous statistical interpretation can be placed on them. The second advantage is that the RMSE procedure does not take into account the uncertainty from the exogenous-variable forecasts, and so it is not possible to use RMSEs to compare models with different degrees of exogeneity. Estimating the uncertainty from the error terms and coefficient estimates is a straightforward exercise in stochastic simulation, for which there is now a fairly large literature.2 The uncertainty from the exogenous variables can also be estimated by means of stochastic simulation, although, as will be discussed, before doing this some assumption about the uncertainty of the exogenous variables themselves must be made. Estimating the uncertainty from the possible misspecification of the model is the most difficult and costly part of the method, and it rests on a strong “constancy” assumption. Although, as will be seen, this assumption is quite restrictive, some assumption of this kind is needed if comparisons across models are to be made. An assumption like this is, for ex-